Image processing apparatus, image processing method, and computer-readable recording device

ABSTRACT

An image processing apparatus includes: an imaging distance estimating unit configured to estimate an imaging distance to a subject shown in an image; an examination region setting unit configured to set an examination region in the image such that an index indicating a spread of a distribution of imaging distances to the subject shown in the examination region is within a given range; and an abnormal structure identifying unit configured to identify whether or not a microstructure of the subject shown in the examination region is abnormal, by using texture feature data that enables identification of an abnormality in the microstructure of the subject shown in the examination region, the texture feature data being specified according the examination region.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of PCT international application Ser.No. PCT/JP2013/064332 filed on May 23, 2013 which designates the UnitedStates, incorporated herein by reference, and which claims the benefitof priority from Japanese Patent Application No. 2012-133187, filed onJun. 12, 2012, incorporated herein by reference.

BACKGROUND

1. Technical Field

The disclosure relates to an image processing apparatus, an imageprocessing method, and a computer-readable recording device, foridentifying an abnormal part region from an image acquired by imaginginside of a lumen of a living body.

2. Related Art

As image processing on an image acquired by imaging inside of a lumen ofa living body by a medical observation apparatus, such as an endoscopeor a capsule endoscope (hereinafter, referred to as “endoscope image” or“intraluminal image”, or simply as “image), a technique is disclosed inJapanese Patent Application Laid-open No. 2005-192880, for example,which is for detecting, from an image, an abnormal part based on amicrostructure of a mucosal surface or a blood stream form. In thistechnique, after extracting, from an endoscope image, an image composedof a green (G) component including a lot of information related to amicrostructure of a mucosa or to a blood stream image, feature datanumerically expressing a pixel value pattern of the mucosal surface arecalculated, and whether or not a subject shown in the image is normal orabnormal is identified by using the feature data and a lineardiscriminant function generated beforehand. As the feature data, forexample, shape feature data (area, groove width, peripheral length,circularity, branch point, edge point, branching rate, or the like; forexample, see Japanese Patent No. 2918162) of a region extracted bybinarizing an image of a particular spatial frequency component, orfeature data based on spatial frequency analysis using Gabor filter orthe like (for example, see Japanese Patent Application Laid-open No.2002-165757) are used. Further, the linear discriminant function isgenerated beforehand with teacher data, which are feature datacalculated from images of normal and abnormal findings.

SUMMARY

In some embodiments, an image processing apparatus includes: an imagingdistance estimating unit configured to estimate an imaging distance to asubject shown in an image; an examination region setting unit configuredto set an examination region in the image such that an index indicatinga spread of a distribution of imaging distances to the subject shown inthe examination region is within a given range; and an abnormalstructure identifying unit configured to identify whether or not amicrostructure of the subject shown in the examination region isabnormal, by using texture feature data that enables identification ofan abnormality in the microstructure of the subject shown in theexamination region, the texture feature data being specified accordingthe examination region.

In some embodiments, an image processing method includes: an imagingdistance estimating step of estimating an imaging distance to a subjectshown in an image; an examination region setting step of setting anexamination region in the image such that an index indicating a spreadof a distribution of imaging distances to the subject shown in theexamination region is within a given range; and an abnormal structureidentifying step of identifying whether or not a microstructure of thesubject shown in the examination region is abnormal, by using texturefeature data that enables identification of an abnormality in themicrostructure of the subject shown in the examination region, thetexture feature data being specified according the examination region.

In some embodiments, a computer-readable recording device with anexecutable program stored thereon is provided. The program instructs aprocessor to perform: an imaging distance estimating step of estimatingan imaging distance to a subject shown in an image; an examinationregion setting step of setting an examination region in the image suchthat an index indicating a spread of a distribution of imaging distancesto the subject shown in the examination region is within a given range;and an abnormal structure identifying step of identifying whether or nota microstructure of the subject shown in the examination region isabnormal, by using texture feature data that enables identification ofan abnormality in the microstructure of the subject shown in theexamination region, the texture feature data being specified accordingthe examination region.

The above and other features, advantages and technical and industrialsignificance of this invention will be better understood by reading thefollowing detailed description of presently preferred embodiments of theinvention, when considered in connection with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of an imageprocessing apparatus according to a first embodiment of the presentinvention;

FIG. 2 is a flow chart illustrating operations of the image processingapparatus illustrated in FIG. 1;

FIG. 3A is a schematic diagram illustrating how inside of a lumen isimaged by an endoscope;

FIG. 3B is a schematic diagram illustrating an intraluminal imagecaptured by the endoscope;

FIG. 4 is a flow chart illustrating in detail a process executed by animaging distance estimating unit illustrated in FIG. 1;

FIG. 5 is a flow chart illustrating in detail a process executed by anexamination region setting unit illustrated in FIG. 1;

FIG. 6 is a schematic diagram illustrating a method of setting anexamination region;

FIG. 7 is a flow chart illustrating in detail a process executed by anabnormal structure identifying unit illustrated in FIG. 1;

FIG. 8 is a block diagram illustrating a configuration of a calculatingunit included in an image processing apparatus according to a modifiedexample 1-1;

FIG. 9 is a flow chart illustrating in detail a process executed by anexamination region setting unit illustrated in FIG. 8;

FIG. 10 is a schematic diagram illustrating the process executed by theexamination region setting unit illustrated in FIG. 8;

FIG. 11 is a flow chart illustrating in detail a process executed by anabnormal structure identifying unit illustrated in FIG. 8;

FIG. 12 is a block diagram illustrating a configuration of a calculatingunit included in an image processing apparatus according to a modifiedexample 1-2;

FIG. 13 is a flow chart illustrating in detail a process executed by anexamination region setting unit illustrated in FIG. 12;

FIG. 14 is a flow chart illustrating in detail a process executed by anabnormal structure identifying unit illustrated in FIG. 12;

FIG. 15 is a block diagram illustrating a configuration of an imageprocessing apparatus according to a second embodiment of the presentinvention;

FIG. 16 is a flow chart illustrating operations of the image processingapparatus illustrated in FIG. 15;

FIG. 17 is a block diagram illustrating a configuration of an imageprocessing apparatus according to a third embodiment of the presentinvention;

FIG. 18 is a flow chart illustrating in detail a process executed by anexamination region setting unit illustrated in FIG. 17;

FIG. 19 is a schematic diagram illustrating the process executed by theexamination region setting unit illustrated in FIG. 17;

FIG. 20 is a flow chart illustrating in detail a process executed by anabnormal structure identifying unit illustrated in FIG. 17;

FIG. 21 is a schematic diagram illustrating intensity characteristics offrequency components according to imaging distance in an intraluminalimage;

FIG. 22 is a block diagram illustrating a configuration of an imageprocessing apparatus according to a fourth embodiment of the presentinvention;

FIG. 23 is a flow chart illustrating operations of the image processingapparatus illustrated in FIG. 22;

FIG. 24 is a flow chart illustrating in detail a process executed by anexamination region setting unit illustrated in FIG. 22;

FIG. 25 is a schematic diagram illustrating the process executed by theexamination region setting unit illustrated in FIG. 22;

FIG. 26 is a block diagram illustrating a configuration of an imageprocessing apparatus according to a fifth embodiment of the presentinvention;

FIG. 27 is a flow chart illustrating operations of the image processingapparatus illustrated in FIG. 26;

FIG. 28 is a flow chart illustrating in detail a process executed by anexamination region deforming unit illustrated in FIG. 26;

FIG. 29 is a schematic diagram illustrating a concept of the processexecuted by the examination region deforming unit illustrated in FIG.26;

FIG. 30 is a block diagram illustrating a configuration of anexamination region deforming unit according to a modified example 5-1;

FIG. 31 is a flow chart illustrating in detail a process executed by theexamination region deforming unit illustrated in FIG. 30;

FIG. 32A is a schematic diagram illustrating a concept of the processexecuted by the examination region deforming unit illustrated in FIG.30; and

FIG. 32B is a schematic diagram illustrating the process executed by theexamination region deforming unit illustrated in FIG. 30.

DETAILED DESCRIPTION

Hereinafter, an image processing apparatus, an image processing method,and an image processing program according to embodiments of the presentinvention will be described with reference to the drawings. The presentinvention is not to be limited by these embodiments. The same referencesigns are used to designate the same elements throughout the drawings.

First Embodiment

FIG. 1 is a block diagram illustrating a configuration of an imageprocessing apparatus according to a first embodiment of the presentinvention. An image processing apparatus 1 according to the firstembodiment, as an example, is an apparatus that performs imageprocessing for identifying an abnormality in a microstructure of amucosal surface with respect to an intraluminal image (hereinafter,simply referred to as “image”) acquired by imaging inside a lumen of aliving body by using an endoscope or a capsule endoscope (hereinafter,these will both be simply referred to as “endoscope”). The intraluminalimage is normally a color image having given (for example,256-gradation) pixel levels (pixel values) for red (R), green (G), andblue (B) wavelength components (color components) at each pixelposition.

As illustrated in FIG. 1, the image processing apparatus 1 includes: acontrol unit 10 that controls operations of the overall image processingapparatus 1; an image acquiring unit 20 that acquires image datacorresponding to the image captured by the endoscope; an input unit 30that receives an input signal input from outside; a display unit 40 thatperforms various displays; a recording unit 50 that stores therein theimage data acquired by the image acquiring unit 20 and various programs;and a calculating unit 100 that executes specified image processing onthe image data.

The control unit 10 is realized by hardware, such as a CPU, and byreading the various programs recorded in the recording unit 50, performstransfer or the like of instructions and data to respective unitsforming the image processing apparatus 1 according to the image datainput from the image acquiring unit 20 and an operation signal and thelike input from the input unit 30 and comprehensively controls theoperations of the overall image processing apparatus 1.

The image acquiring unit 20 is configured as appropriate according to amode of a system including the endoscope. For example, if a portablerecording medium is used in transfer of image data to and from thecapsule endoscope, the image acquiring unit 20 is configured of a readerdevice, to which this recording medium is detachably attached, and whichreads the recorded image data of the image. Further, if a server tostore therein the image data of the image captured by the endoscope isto be arranged, the image acquiring unit 20 is configured of acommunication device or the like connected to the server and performsdata communication with the server to acquire the image data. Or, theimage acquiring unit 20 may be configured of an interface device or thelike that inputs an image signal from the endoscope via a cable.

The input unit 30 is realized by an input device, such as a key boardand a mouse, a touch panel, or various switches, and outputs thereceived input signal to the control unit 10.

The display unit 40 is realized by a display device, such as an LCD oran EL display, and under the control by the control unit 10, displaysvarious screens including the intraluminal image.

The recording unit 50 is realized by: various IC memories, such as a ROMand a RAM like rewritable flash memories; a built-in hard disk or a harddisk connected via a data communication terminal; an informationrecording device such as a CD-ROM and a reading device thereof; or thelike. The recording unit 50 stores therein the image data acquired bythe image acquiring unit 20, as well as a program for causing the imageprocessing apparatus 1 to operate and causing the image processingapparatus 1 to execute various functions, data used during the executionof this program, and the like. Specifically, the recording unit 50stores therein an image processing program 51 for identifying anabnormality in a microstructure of a mucosal surface shown in an image,various information used during execution of this program, and the like.

The calculating unit 100 is realized by hardware such as a CPU, and byreading the image processing program 51, performs the image processingon the intraluminal image and executes various calculating processes foridentifying an abnormality in a microstructure of a mucosal surface.

Next, a detailed configuration of the calculating unit 100 will bedescribed.

As illustrated in FIG. 1, the calculating unit 100 includes: an imagingdistance estimating unit 110 that estimates an imaging distance to asubject shown in an image; an examination region setting unit 120 thatsets an examination region in the image, such that an index indicating aspread of a distribution of imaging distance of the subject shown in theexamination region is within a given range; and an abnormal structureidentifying unit 130 that identifies, by using texture feature dataenabling identification of an abnormality in a microstructure of thesubject shown in the examination region, whether or not themicrostructure of the subject shown in the examination region isabnormal, the texture feature data being specified according to theexamination region.

The texture in the image processing herein is a repeated brightnesspattern (reference: “Digital Image Processing” by CG-ARTS Society, page192 (“Texture of Region”)). In this first embodiment, a particularspatial frequency component is used as the texture feature datanumerically expressing a feature of the texture.

Of the configuration of the calculating unit 100, the imaging distanceestimating unit 110 includes a low absorbance wavelength selecting unit111 that selects, from pixel values (respective values of R-component,G-component, and B-component) of each pixel in the image, a value of theR-component (hereinafter, referred to as “R-component value”)corresponding to a low absorbance wavelength component, which is awavelength component having the lowest degree of absorption orscattering in the living body, and the imaging distance estimating unit110 estimates the imaging distance to the subject shown in the image,based on the R-component value.

The examination region setting unit 120 includes a candidate regionsetting unit 121 that sets an examination candidate region in the imageand a region determining unit 122 that determines the examination regionbased on imaging distance information of the subject shown in the setexamination candidate region. Of these, the region determining unit 122includes an imaging distance range calculating unit 122 a thatcalculates a distribution range of imaging distance to the subject shownin the examination candidate region, and the region determining unit 122determines the examination candidate region with the distribution rangeequal to or less than a given threshold value to be the examinationregion.

The abnormal structure identifying unit 130 includes a particularfrequency component calculating unit 131 that calculates a particularspatial frequency component in the examination region and a statisticalclassification unit 132 that performs statistical classification basedon the particular spatial frequency component. The particular spatialfrequency component will be described later.

Next, operations of the image processing apparatus 1 will be described.FIG. 2 is a flow chart illustrating operations of the image processingapparatus 1. In this first embodiment, a mucosa is assumed to be shownin most of the regions of the intraluminal image and hereinafter, aprocess with respect to a mucosa area will be described.

First, at Step S11, the calculating unit 100 acquires, by reading theimage data recorded in the recording unit 50, an intraluminal image tobe processed.

FIG. 3A is a schematic diagram illustrating how the inside of a lumen isimaged by an endoscope. Further, FIG. 3B is a schematic diagramillustrating the intraluminal image captured by the endoscope. Anendoscope 6 performs imaging with a visual field “V” directed in alongitudinal direction of a lumen 7. Thus, in an image “M” having amucosal surface 8, which is a subject shown therein, an image region m₁having a near view v₁ taken therein and an image region m₂ having adistant view v₂ shown therein coexist. Therefore, due to a difference inimaging distance “r”, a difference in image quality is caused betweenthe image region m₁ and the image region m₂. Hereinafter, as an example,the image “M” will be described as a target to be processed.

At subsequent Step S12, the imaging distance estimating unit 110estimates the imaging distance “r” to the mucosal surface 8 shown in theimage. FIG. 4 is a flow chart illustrating in detail a process executedby the imaging distance estimating unit 110.

At Step S121, the low absorbance wavelength selecting unit 111 selectsthe low absorbance wavelength component, which is the wavelengthcomponent having the lowest degree of absorption or scattering in theliving body. This is for suppressing influence of absorption orscattering of light by a blood vessel or the like near the mucosalsurface 8 and for acquiring pixel values reflecting the imaging distance“r” between the endoscope 6 and the mucosal surface 8 most well. In animage composed of respective components of R, G, and B, the R-componentis the most away from an absorption band of blood and is also thecomponent of the longest wavelength, and thus is hard to be affected bythe absorption or scattering in the living body. Therefore, in thisfirst embodiment, the R-component is selected.

At subsequent Step S122, the imaging distance estimating unit 110estimates, based on the R-component value selected as the low absorbancewavelength component, the imaging distance “r” when the mucosal surface8 is assumed to be a uniform diffuser, by using next Equation (1).

$\begin{matrix}{r = \sqrt{\frac{I \times K \times \cos \; \theta}{L}}} & (1)\end{matrix}$

In Equation (1), the symbol “I” is the emission power of a light sourcein the endoscope 6 and a measured value that has been measuredbeforehand is applied. The symbol “K” is a diffused reflectioncoefficient of the mucosal surface 8 and an average value is measuredbeforehand and applied. The symbol θ is an angle formed by a normalvector of the mucosal surface 8 and a vector from the mucosal surface 8to the light source (endoscope 6). The angle θ is actually a valueindividually determined by a positional relation between the lightsource provided at a distal end of the endoscope 6 and the mucosalsurface 8, but an average value is set beforehand and applied. Thesymbol “L” is the R-component value of a pixel having an estimationtarget region on the mucosal surface 8 at the imaging distance “r” takentherein.

At Step S13 subsequent to Step S12, the examination region setting unit120 sets an examination region in the image such that an indexindicating a spread of a distribution of the imaging distance “r” of thesubject shown in the examination region is within a given range. In thisfirst embodiment, as the index indicating the spread of thedistribution, a distribution range of the imaging distance “r” is used.FIG. 5 is a flow chart illustrating in detail a process executed by theexamination region setting unit 120. Further, FIG. 6 is a schematicdiagram illustrating the process executed by the examination regionsetting unit 120.

At Step S131, the candidate region setting unit 121 sets, as illustratedin FIG. 6, each of rectangular regions of a given size acquired bydividing the image “M” in a grid pattern, as an examination candidateregion CA_(i) (i=1, 2, . . . , n).

At subsequent Step S132, the imaging distance range calculating unit 122a calculates the distribution range of the imaging distance “r” to themucosal surface 8 shown in each examination candidate region CA_(i).Specifically, in each examination candidate region CA_(i), a differenceΔr (Δr=r_(max)−r_(min)) between the minimum value r_(min) and themaximum value r_(max) of the imaging distance “r” calculated from theR-component value of each pixel is found as the distribution range.

Further, at Step S133, the region determining unit 122 determines theexamination candidate region with the distribution range Δr equal to orless than a given threshold value to be an examination region. In thisfirst embodiment, an examination region EA_(j) (j=1, 2, . . . , m; m≦n)illustrated by a bold frame in FIG. 6 is assumed to be determined.

At Step S14 subsequent to Step S13, the abnormal structure identifyingunit 130 identifies whether or not a surface of the subject shown in thedetermined examination region, that is, a microstructure of the mucosalsurface 8, is abnormal. FIG. 7 is a flow chart illustrating in detail aprocess executed by the abnormal structure identifying unit 130.

At Step S141, the particular frequency component calculating unit 131calculates, for each examination region EA_(j), an intensity of aparticular spatial frequency component of each of wavelength components(R-component, G-component, and B-component) forming the image “M”, foreach pixel. The particular spatial frequency component is a spatialfrequency component enabling identification of presence or absence of anabnormality in the microstructure of the mucosal surface 8 shown in theimage “M”, and is set beforehand based on teacher data or the like.

Calculation of the particular spatial frequency component may berealized by applying a known band pass filter to each wavelengthcomponent of the examination region EA_(j) (reference: “Digital ImageProcessing” by CG-ARTS Society, page 136 (“Band Pass Filter”) and page141 (“LOG Filter”)). In the first embodiment, band pass filtering(calculation of the particular spatial frequency component) is notperformed with respect to pixels (end portion pixels) positioned at anend portion of the examination region EA_(j). The reason for that isbecause if a particular spatial frequency component of an end portionpixel of the examination region EA_(j) is calculated, a pixel outsidethe examination region EA_(j) needs to be used, but like for theexamination region EA₃, for example, if the examination region EA_(j) ispositioned at an end portion of the image “M”, the outside of theexamination region EA_(j) is thus outside of the image “M” and there maybe no pixels there. Further, even if there is a pixel outside theexamination region EA_(j), a value of that pixel outside the examinationregion EA_(j) may be largely different from a pixel value of a pixelinside the examination region EA_(j).

At Step S142, the statistical classification unit 132 calculates anaverage intensity of the particular spatial frequency component amongthe pixels in each examination region EA_(j) for each wavelengthcomponent and generates a feature vector “x” having these averageintensities as its components. In the first embodiment, calculation isperformed with respect to the three wavelength components of R, G, andB, and thus the number of components of the feature vector “x” is three(that is, a matrix of three rows and one column).

If the microstructure of the mucosal surface is abnormal, a differencein intensity from that of a normal microstructure is generated, in aparticular spatial frequency component of an intermediate band excludinga low frequency component representing a shape of the mucosal surfaceand a high frequency component representing imaging noise.

Therefore, at Step S143, the abnormal structure identifying unit 130performs classification of whether or not each examination region EA_(j)is an abnormal region based on a discriminant function for abnormalregion generated beforehand and on the feature vector generated from theparticular spatial frequency components. In an actual process, aclassification index P(x) based on a probability model expressed byEquation (2) is calculated, and if this value is equal to or greaterthan a threshold value, that examination region EAj is classified as anabnormal region.

$\begin{matrix}{{P(x)} = {\frac{1}{( {2\pi} )^{k/2} \times {Z}^{1/2}}\exp \{ {( {x - \mu} )^{t} \times ( {- \frac{1}{2}} )Z^{- 1} \times ( {x - \mu} )} \}}} & (2)\end{matrix}$

In Equation (2), the symbol μ is an average vector (three rows and onecolumn) of feature vectors in samples of a plurality of abnormal regionsacquired beforehand. The symbol “Z” is a variance-covariance matrix(three rows and three columns) in the samples of the plurality ofabnormal regions acquired beforehand. The symbol |Z| is a determinant ofthe variance and covariance matrix. The symbol Z⁻¹ is an inverse matrixof the variance and covariance matrix. The symbol “k” is adimensionality of the feature vector “x” and in the first embodiment,k=3.

In this first embodiment, the classification method for an abnormalregion using the probability model has been described, but as long asclassification of whether each examination region is abnormal or normalis possible, any method other than the one described above may be used.For example, classification may be performed by a method based on afeature space distance from a representative feature vector, a method ofsetting a classification boundary in a feature space, or the like.

At Step S15, the calculating unit 100 outputs a result of theabnormality identification in Step S14, causes the display unit 40 todisplay the result and the recording unit 50 to record the result.Thereafter, the process in the image processing apparatus 1 is ended.

As described above, according to the first embodiment, since theexamination region is set in the image such that the index indicatingthe spread of the distribution of the imaging distance of the subjectshown in the examination region is within the given range, and for eachexamination region, whether or not the microstructure of the subjectshown in the examination region is abnormal is identified by using theparticular spatial frequency component enabling identification of anabnormality in the microstructure of the subject shown in theexamination region as the texture feature data; even if a difference inresolution with respect to a microstructure of a subject (mucosa)surface shown in an image is caused due to a difference in imagingdistance, like, for example, between a distant view and a near view, anabnormality in a microstructure of a mucosal surface is able to beidentified accurately.

Modified Example 1-1

Next, a modified example 1-1 of the first embodiment will be described.

FIG. 8 is a block diagram illustrating a configuration of a calculatingunit included in an image processing apparatus according to the modifiedexample 1-1. As illustrated in FIG. 8, a calculating unit 100-1according to the modified example 1-1 includes the imaging distanceestimating unit 110, an examination region setting unit 140, and anabnormal structure identifying unit 150. The configuration andoperations of the imaging distance estimating unit 110 are similar tothose of the first embodiment. Further, the configuration and operationsof the overall image processing apparatus other than the calculatingunit 100-1 are similar to those of the first embodiment.

The examination region setting unit 140 includes a candidate regionsetting unit 141 and a region determining unit 142. Of these, thecandidate region setting unit 141 includes a representative imagingdistance acquiring unit 141 a that acquires a representative imagingdistance to a subject taken at a position where an examination candidateregion is set, and the candidate region setting unit 141 sets theexamination candidate region of a size corresponding to therepresentative imaging distance. Further, the region determining unit142 includes an imaging distance variance calculating unit 142 a thatcalculates a variance of the imaging distance to the subject shown inthe examination candidate region, and the region determining unit 142determines the examination candidate region with the variance equal toor less than a given threshold value to be an examination region.

The abnormal structure identifying unit 150 includes a particularwavelength component selecting unit 151 that selects a particularwavelength component specified according to a degree of absorption orscattering in a living body, a particular frequency componentcalculating unit 152 that calculates a particular frequency component atthe selected wavelength, and a statistical classification unit 153 thatperforms statistical classification based on the particular spatialfrequency component.

Next, operations of the calculating unit 100-1 will be described.

The operations of the calculating unit 100-1 as a whole are similar tothose illustrated in FIG. 2 and the detailed processes in Steps S13 andS14 are different. FIG. 9 is a flow chart illustrating in detail aprocess (Step S13) executed by the examination region setting unit 140.FIG. 10 is a schematic diagram illustrating the process executed by theexamination region setting unit 140. FIG. 11 is a flow chartillustrating in detail a process (Step S14) executed by the abnormalstructure identifying unit 150.

At Step S13 subsequent to Step S12, the examination region setting unit140 sets an examination region in the image “M” such that only a subjectwith an index indicating a spread of a distribution of imaging distanceestimated in Step S12 within a given range is included therein. In thismodified example 1-1, as the index indicating the spread of thedistribution, rather than the distribution range of the imagingdistance, a variance of the imaging distance, which is more stableagainst noise, will be used.

In more detail, as illustrated in FIG. 10, at Step S231, the candidateregion setting unit 141 randomly determines a plurality of centerpositions (x_(i), y_(i)) (i=1, 2, . . . ) of regions where examinationcandidate regions are to be set. In FIG. 10, as an example, three centerpositions (x₁, y₁), (x₂, y₂) and (x₃, y₃) are illustrated.

At subsequent Step S232, the representative imaging distance acquiringunit 141 a acquires an imaging distance to the subject taken at eachcenter position (x_(i),y_(i)). In an image, generally, a subject in anear view portion is taken largely and a subject in a distant viewportion is taken small (see FIG. 3A). Therefore, if an examinationregion is set small as the imaging distance gets longer, a possibilityof being able to suppress the variance of the imaging distance to thesubject included in the examination region to be equal to or less than agiven value is increased.

Thus, at Step S233, the candidate region setting unit 141 sets anexamination candidate region CB_(i) of a size according to the imagingdistance with each center position (x_(i), y_(i)) being defined as thecenter thereof. The shape of the examination region is not particularlylimited, and various shapes are applicable, such as a rectangular shape,and a circular shape. For example, in FIG. 10, for the center position(x₁, y₁) of the near view portion, a comparatively large examinationcandidate region CB′ is set, and for the center position (x₃, y₃) of thedistant view portion, a comparatively small examination candidate regionCB₃ is set. Further, at the center position (x₂, y₂) of the intermediateportion between these, an examination candidate region CB₂ of anintermediate size is set.

At Step S234, the imaging distance variance calculating unit 142 acalculates a variance of a distribution of the imaging distance to thesubject shown in each examination candidate region CB_(i).

At Step S235, the region determining unit 142 determines the examinationcandidate region CB_(i) with a variance equal to or less than a giventhreshold value to be an examination region EB_(i). For example, in FIG.10, the examination candidate regions CB₁ and CB₃ are determined to beexamination regions EB₁ and EB₃.

At Step S14 subsequent to Step S13, the abnormal structure identifyingunit 150 identifies whether or not a microstructure of a mucosal surfaceshown in the examination region EB_(i) is abnormal. In the firstembodiment, an abnormality in the microstructure is identified by usingthe particular spatial frequency components for all of wavelengthcomponents (R-component, G-component, and B-component). However, anabnormality in a microstructure of a mucosal surface is usually causedby a state of formation of the capillary vessels. Therefore, awavelength component near the absorption band of blood demonstrates aprominent change. Accordingly, in this modified example 1-1,identification of an abnormal structure is performed by using, as thetexture feature data, a particular spatial frequency component at aparticular wavelength with a high degree of absorption or scattering ina living body.

In detail, at Step S241, the particular wavelength component selectingunit 151 selects, as the particular wavelength component having a highdegree of absorption or scattering in the living body, for example, theG-component or the B-component.

At subsequent Step S242, the particular frequency component calculatingunit 152 calculates, for each pixel, an intensity of the particularspatial frequency component at the selected wavelength component foreach examination region EB_(i). The particular spatial frequencycomponent is set beforehand based on teacher data or the like.

At Step S243, the statistical classification unit 153 calculates anaverage intensity of the particular spatial frequency component of theselected wavelength component among the pixels and determines a value ofthis average intensity as feature data.

Further, at Step S244, the abnormal structure identifying unit 150performs, based on a discriminant function for an abnormal regiongenerated beforehand and the feature data, classification of whether ornot each examination region EB_(i) is an abnormal region. The processusing the discriminant function is similar to that of the firstembodiment. However, in a calculating formula of the classificationindex P(x) expressed by Equation (2), instead of the feature vector “x”,the feature data calculated in Step S243 are applied. Further, insteadof the average vector μ in Equation (2), an average value of featuredata in samples of a plurality of samples of abnormal regions acquiredbeforehand is applied. Furthermore, instead of the variance-covariancematrix “Z” in Equation (2), a variance in the plurality of samples ofabnormal regions acquired beforehand is applied, and instead of theinverse matrix Z⁻¹, a reciprocal of the variance in the samples isapplied. Moreover, in Equation (2), k=1.

As described above, according to the modified example 1-1, since thesize of the examination candidate region is changed according to theimaging distance, an examination region with an index indicating aspread of a distribution of imaging distance of a subject shown in theexamination region within a given range is able to be set efficiently.As a result, identification of an abnormality in a microstructure for amucosal surface of a wider range becomes possible and accuracy of theidentification of an abnormality in a microstructure is able to beimproved. In addition, by specifying the wavelength to be used in theidentification of an abnormality in the microstructure, an abnormalityin a microstructure accompanying absorption change is able to beidentified accurately.

Modified Example 1-2

Next, a modified example 1-2 of the first embodiment will be described.

FIG. 12 is a block diagram illustrating a configuration of a calculatingunit included in an image processing apparatus according to a modifiedexample 1-2. As illustrated in FIG. 12, a calculating unit 100-2according to the modified example 1-2 includes the imaging distanceestimating unit 110, an examination region setting unit 160, and anabnormal structure identifying unit 170. The configuration andoperations of the imaging distance estimating unit 110 are similar tothose of the first embodiment. Further, the configuration and operationsof the overall image processing apparatus other than the calculatingunit 100-2 are similar to those of the first embodiment.

The examination region setting unit 160 includes a candidate regionsetting unit 161 and the region determining unit 142. The candidateregion setting unit 161 includes a representative imaging distancegradient calculating unit 161 a that calculates a representative imagingdistance gradient of a subject taken at a position where an examinationcandidate region is to be set, and the candidate region setting unit 161sets the examination candidate region of a size corresponding to therepresentative imaging distance gradient. The configuration andoperations of the region determining unit 142 are similar to those ofthe modified example 1-1.

The abnormal structure identifying unit 170 includes aninter-particular-wavelength ratio calculating unit 171 that calculates aratio between particular wavelength components having different degreesof absorption or scattering in a living body, a particular frequencycomponent calculating unit 172 that calculates a particular spatialfrequency component for the calculated ratio between the particularwavelength components, and a statistical classification unit 173 thatperforms statistical classification based on the particular spatialfrequency component.

Next, operations of the calculating unit 100-2 will be described.

The operations of the calculating unit 100-2 as a whole are similar tothose illustrated in FIG. 2 and the detailed processes in Steps S13 andS14 are different. FIG. 13 is a flow chart illustrating in detail aprocess (Step S13) executed by the examination region setting unit 160.FIG. 14 is a flow chart illustrating in detail a process (Step S14)executed by the abnormal structure identifying unit 170. Steps S231,S234, and S235 illustrated in FIG. 13 correspond to those illustrated inFIG. 9.

At Step S232′ subsequent to Step S231 illustrated in FIG. 13, therepresentative imaging distance gradient calculating unit 161 acalculates an imaging distance gradient of a subject taken at eachcenter position (x_(i), y_(i)) determined randomly in the image “M” (seeFIG. 10). In the actual process, a known one dimensional differentialfilter (reference: “Digital Image Processing” by CG-ARTS Society, page114 (“Differential Filter”)) is applied to the imaging distance of thesubject taken at each pixel position to find an absolute value of thecalculated value.

The larger the imaging distance gradient is, the larger the range of theimaging distance in a given region becomes. Therefore, if an examinationregion is set smaller as the imaging distance gradient at the positionwhere an examination region is to be set gets larger, a possibility ofbeing able to suppress the variance of the imaging distance to thesubject included in the examination region to be equal to or less than agiven value is increased.

Thus, at Step S233′, the candidate region setting unit 161 determines asize of the examination candidate region according to the imagingdistance gradient at each center position (x_(i),y_(i)) and sets anexamination candidate region of the size according to the imagingdistance gradient with each center position (x_(i), y_(i)) being definedas the center thereof.

Steps S234 and S235 thereafter are similar to those of the modifiedexample 1-1.

At Step S14 subsequent to Step S13, the abnormal structure identifyingunit 170 identifies whether or not a microstructure of a mucosal surfaceshown in the examination region is abnormal. In the modified example1-1, an abnormality in the microstructure is identified by using theparticular spatial frequency component at the particular wavelength witha high degree of absorption or scattering in the living body. However,the pixel value change in a microstructure shown in an image isinfluenced by the imaging distance, and in the distant view portion, thechange is small, and in the near view portion, the change is large.Therefore, the average intensity of the particular spatial frequencycomponent calculated in the modified example 1-1 includes the pixelvalue change corresponding to the imaging distance and if identificationof an abnormal structure by using the same discriminant function isperformed, there is a possibility that the identification accuracy maybe reduced. Accordingly, in this modified example 1-2, in order tosuppress the influence by the pixel value change associated with theimaging distance, a ratio between particular wavelength componentshaving degrees of absorption or scattering in a living body differentfrom each other is calculated, and an abnormal structure is identifiedby using, as the texture feature data, a particular spatial frequencycomponent at that ratio.

In detail, at Step S241′ illustrated in FIG. 14, theinter-particular-wavelength ratio calculating unit 171 calculates, basedon a pixel value of each pixel in an examination region, for example,G/R or the like, as the ratio between the particular wavelengthcomponents with the degrees of absorption or scattering in the livingbody different from each other. Hereinafter, the ratio calculatedthereby will be referred to as “inter-wavelength ratio”.

At subsequent Step S242′, the particular frequency component calculatingunit 172 calculates, for each pixel in each examination region, anintensity of the particular spatial frequency component of theinter-wavelength ratio. The particular spatial frequency component isset beforehand based on teacher data or the like.

At Step S243′, the statistical classification unit 173 calculates anaverage intensity of the particular spatial frequency component of theinter-wavelength ratio among the pixels, and determines a value of thisaverage intensity as feature data.

Further, at Step S244, the abnormal structure identifying unit 170performs, based on a discriminant function for an abnormal regiongenerated beforehand and the feature data, classification of whether ornot each examination region is an abnormal region. The details of thisprocess are similar to those of the modified example 1-1.

As described above, according to the modified example 1-2, because thesize of the examination region is changed according to the imagingdistance gradient at the position where the examination region is to beset, an examination region with an index indicating a spread of adistribution of imaging distance within a given range is able to be setefficiently. As a result, identification of an abnormality in amicrostructure for a mucosal surface of a wider range becomes possibleand accuracy of the identification of an abnormality in themicrostructure is able to be improved. Further, by using theinter-wavelength ratio upon the identification of an abnormality in themicrostructure, an intensity change of a particular spatial frequencycomponent caused according to imaging distance is able to be suppressed,and an abnormality in a microstructure is able to be identifiedaccurately.

Second Embodiment

Next, a second embodiment of the present invention will be described.

FIG. 15 is a block diagram illustrating a configuration of an imageprocessing apparatus according to the second embodiment of the presentinvention. As illustrated in FIG. 15, an image processing apparatus 2according to the second embodiment includes a calculating unit 200,instead of the calculating unit 100 illustrated in FIG. 1. Theconfigurations and operations of the respective units of the imageprocessing apparatus 2 other than the calculating unit 200 are similarto those of the first embodiment.

The calculating unit 200 includes: a non-examination region excludingunit 210 that excludes, from a target to be processed, a region(non-examination target region) not to be identified for an abnormalityin the image; the imaging distance estimating unit 110; the examinationregion setting unit 120; a repeat control unit 220 that performs controlof causing setting of an examination region for a region where anexamination region has not been set to be repeatedly executed; and theabnormal structure identifying unit 130. Of these, the configurationsand operations of the imaging distance estimating unit 110, theexamination region setting unit 120, and the abnormal structureidentifying unit 130 are similar to those of the first embodiment.Instead of the examination region setting unit 120 and the abnormalstructure identifying unit 130, the examination region setting unit 140and the abnormal structure identifying unit 150 according to themodified example 1-1 or the examination region setting unit 160 and theabnormal structure identifying unit 170 according to the modifiedexample 1-2 may be applied.

Next, operations of the image processing apparatus 2 will be described.FIG. 16 is a flow chart illustrating the operations of the imageprocessing apparatus 2.

First, at Step S21, the calculating unit 200 reads the image datarecorded in the recording unit 50 to thereby acquire an intraluminalimage to be processed.

At subsequent Step S22, the non-examination region excluding unit 210identifies, based on color information, frequency information, shapeinformation, and the like acquirable from the image, a non-examinationregion, such as a dark part, a bright part, a residue, a bubble, or thelike, and excludes the non-examination region from the examinationtarget.

An intraluminal image contains, other than a mucosa region to beexamined, a region where a deep part of a lumen is shown (dark part), ahalation region which is mirror-reflected from a surface of a subject(bright part), a region where a residue or a bubble is shown, and thelike. If these regions are included in an examination region, accuracyof identification of an abnormality in a microstructure is reduced.Thus, the non-examination region excluding unit 210 extracts, from theimage, a region where a bright part, a dark part, a residue, a bubble,or the like is shown and excludes the region as a non-examinationregion. These non-examination regions may be extracted by various knownmethods. For example, a dark part may be extracted by extracting a blackregion, based on color feature data that is based on color information(respective values of R-component, G-component, and B-component, or thelike) of each pixel in an image, and identifying, based on a directionof a pixel value change around this black region, whether or not theblack region is a dark part (reference: Japanese Patent ApplicationLaid-open No. 2011-234931). Further, a bright part may be extracted, forexample, by extracting a white region, based on the color feature dataof each pixel in the image, and identifying, based on a pixel valuechange around a boundary of this white region, whether or not the whiteregion is a halation region (same as above). A residue may be extractedby detecting, based on the color feature data of each pixel in theimage, a residue candidate region likely to be a non-mucosa region, andidentifying, based on a positional relation between this residuecandidate region and a structural edge in the image, whether or not theresidue candidate region is a mucosa region. A bubble may be extractedby extracting an edge from the image and calculating a correlation valuebetween a bubble model set beforehand based on a feature of a bubbleimage and the extracted edge (reference: Japanese Patent ApplicationLaid-open No. 2007-313119).

At Step S23, the imaging distance estimating unit 110 estimates animaging distance to a subject shown in the image. This estimatingprocess is similar to that of the first embodiment (see Step S12 of FIG.2).

At Step S24, the examination region setting unit 120 sets an examinationregion in the image such that an index indicating a spread of adistribution of imaging distance of the subject shown in the examinationregion is within a given range. This process of setting the examinationregion is similar to that of the first embodiment (see Step S13 of FIG.2).

At Step S25, the repeat control unit 220 determines whether an area ofan unexamined region, which is a region where an examination region hasnot been set yet, is smaller than a given threshold value (thresholdvalue “A”) or whether or not the number of times an examination regionhas been set so far is larger than a given threshold value (thresholdvalue “N”). If examination regions in an image have not been setsufficiently, there is a possibility that accuracy of identification ofan abnormality in a microstructure may be reduced. Accordingly, if thearea of the unexamined region is equal to or greater than the thresholdvalue “A” and the number of times an examination region has been set isequal to or less than the threshold value “N” (Step S25: No), the repeatcontrol unit 220 determines that setting of an examination region isneeded further, and sets the size of an examination region to be smallerthan that set last time (Step S26). The repeat control unit 220 proceedsto Step S24 and causes the examination region setting unit 120 toexecute the setting of an examination region again. By decreasing thesize of an examination region like this, a range of imaging distanceincluded in one examination region is often narrowed, and thus apossibility of regions settable as examination regions being increasedin an image becomes higher.

On the contrary, if the area of the unexamined region is less than thethreshold value “A” or the number of times an examination region hasbeen set is greater than the threshold value “N” (Step S25: Yes), therepeat control unit 220 determines that further setting of anexamination region is not necessary, proceeds to Step S27, and causesthe abnormal structure identifying unit 130 to execute identification ofan abnormality in a microstructure. This process of identifying anabnormality in the microstructure is similar to that of the firstembodiment (see Step S14 of FIG. 2).

Further, at Step S28, the calculating unit 200 outputs a result of theabnormality identification (see Step S15 of FIG. 2).

As described above, according to the second embodiment, since theexamination region is set by excluding the non-examination regionbeforehand, an abnormality in a microstructure is able to be identifiedaccurately. Further, by repeating the setting of an examination region,identification of an abnormality in a microstructure for a mucosalsurface of a wider range becomes possible and accuracy of theidentification of an abnormality in the microstructure is able to beimproved. Further, by decreasing the size of an examination region everytime the process is repeated, an abnormality in a microstructure is ableto be identified for a mucosal surface of a wider range and accuracy ofthe identification of an abnormality in the microstructure is able to beimproved.

Third Embodiment

Next, a third embodiment of the present invention will be described.

FIG. 17 is a block diagram illustrating a configuration of an imageprocessing apparatus according to the third embodiment of the presentinvention. As illustrated in FIG. 17, an image processing apparatus 3according to the third embodiment includes a calculating unit 300,instead of the calculating unit 100 illustrated in FIG. 1. Theconfigurations and operations of the respective units of the imageprocessing apparatus 3 other than the calculating unit 300 are similarto those of the first embodiment.

The calculating unit 300 includes the imaging distance estimating unit110, an examination region setting unit 310, and an abnormal structureidentifying unit 320. Of these, the configuration and operations of theimaging distance estimating unit 110 are similar to those of the firstembodiment.

The examination region setting unit 310 includes a level classificationunit 311 that classifies values of imaging distance into one level or aplurality of levels; a region dividing unit 312 that divides, for eachregion where the subject at imaging distance classified into the samelevel is shown, the image into one region or a plurality of regions, andthe examination region setting unit 310 sets each of the one region orplurality of regions acquired by the region dividing unit 312 as anindividual examination region.

The abnormal structure identifying unit 320 includes a representativeimaging distance acquiring unit 321 that acquires a representativeimaging distance to a subject shown in an examination region, aparticular frequency component calculating unit 322 that calculates aparticular spatial frequency component according to the representativeimaging distance, and a statistical classification unit 323 thatperforms statistical classification, based on the particular spatialfrequency component.

Next, operations of the image processing apparatus 3 will be described.

The operations of the image processing apparatus 3 as a whole aresimilar to those illustrated in FIG. 2 and the detailed processes atSteps S13 and S14 are different. FIG. 18 is a flow chart illustrating indetail a process (Step S13) executed by the examination region settingunit 310. FIG. 19 is a schematic diagram illustrating the processexecuted by the examination region setting unit 310. FIG. 20 is a flowchart illustrating in detail a process (Step S14) executed by theabnormal structure identifying unit 320. FIG. 21 is a schematic diagramillustrating intensity characteristics of frequency components accordingto imaging distance in an intraluminal image.

At Step S13 subsequent to Step S12, the examination region setting unit310 sets an examination region in the image “M” such that an indexindicating a spread of a distribution of imaging distance of a subjectshown in the examination region is within a given range.

In detail, first, at Step S331 illustrated in FIG. 18, the levelclassification unit 311 classifies values of the imaging distances tothe subject shown in the image “M” into a plurality of given levels.Each level is set such that a range of the imaging distance is equal toor less than a given value.

At subsequent Step S332, the region dividing unit 312 divides the image“M” into each region where the subject at the same level of imagingdistance is shown. For example, in FIG. 19, the image “M” is dividedinto four divided regions B1 to B4 correspondingly with a level R1 wherethe imaging distance is equal to or less than r1, a level R2 where theimaging distance is in a range of r1 to r2, a level R3 where the imagingdistance is in a range of r2 to r3, and a level R4 where the imagingdistance is equal to or greater than r3.

At Step S333, the examination region setting unit 310 sets the dividedregions B1 to B4 corresponding to the respective levels R1 to R4 asindividual examination regions.

At Step S14 subsequent to Step S13, the abnormal structure identifyingunit 320 identifies whether or not a microstructure of a mucosal surfaceshown in the examination region is abnormal.

In detail, at Step S341 illustrated in FIG. 20, the representativeimaging distance acquiring unit 321 acquires a representative imagingdistance to the subject in each of the examination regions (dividedregions) B1 to B4. Examples of the representative imaging distanceinclude an average value of the imaging distances to the subjectincluded in the examination regions B1 to B4 or an imaging distance atcoordinates of gravity center of the examination regions B1 to B4.

At subsequent Step S342, the particular frequency component calculatingunit 322 identifies, for each of the examination regions B1 to B4,according to the representative imaging distance, a spatial frequencycomponent to be used in identification of an abnormality. As describedabove, in an intraluminal image captured by an endoscope, differentresolutions of a microstructure of a mucosal surface are obtainedaccording to the imaging distance. Specifically, the longer the imagingdistance is, the lower the resolution becomes. Therefore, for example,as illustrated in FIG. 21, if the imaging distance is short (near view)and a particular spatial frequency enabling identification of anabnormality in a microstructure is f₁, a spatial frequency enablingidentification of an abnormality in the same microstructure shifts to ahigher frequency side as the imaging distance gets longer (distant view:spatial frequency f₂).

Accordingly, in this third embodiment, according to the representativeimaging distance of each of the examination regions B1 to B4, aparticular spatial frequency component to be used as texture featuredata upon identification of an abnormality in a microstructure ischanged, in order to achieve improvement in accuracy of theidentification and increase in the efficiency of the process.Specifically, the longer the imaging distance is, the higher theparticular spatial frequency component is made, to thereby enabledetection for a finer structure. The shorter the imaging distance is,the lower the particular spatial frequency component is made, to therebysuppress the amount of calculation. The particular spatial frequencycomponent according to the imaging distance is set beforehand based onteacher data or the like.

At Step S343, the particular frequency component calculating unit 322calculates, for each pixel in each of the examination regions B1 to B4,an intensity of the particular spatial frequency component according tothe imaging distance for each wavelength component forming the image.The process of calculating the intensity of the particular spatialfrequency component is similar to that of the first embodiment (see StepS141 of FIG. 7).

Subsequent Steps S142 and S143 are similar to those of the firstembodiment.

As described above, according to the third embodiment, without repeatingthe process of setting or determining an examination candidate region orthe like, an examination region over a wide range in an image is able tobe set efficiently. As a result, accurate identification of anabnormality in a microstructure becomes possible. Further, since thespatial frequency component to be used in the identification of anabnormality in a microstructure is specified based on the representativeimaging distance to the subject shown in the examination region,accurate identification of an abnormality in a microstructure becomespossible regardless of imaging distance and efficiency of thecalculation process is able to be increased.

Fourth Embodiment

Next, a fourth embodiment of the present invention will be described.

FIG. 22 is a block diagram illustrating a configuration of an imageprocessing apparatus according to the fourth embodiment of the presentinvention. As illustrated in FIG. 22, an image processing apparatus 4according to the fourth embodiment includes a calculating unit 400,instead of the calculating unit 100 illustrated in FIG. 1. Theconfigurations and operations of the respective units of the imageprocessing apparatus 4 other than the calculating unit 400 are similarto those of the first embodiment.

The calculating unit 400 includes the imaging distance estimating unit110, an examination region setting unit 410, the repeat control unit220, and the abnormal structure identifying unit 130. Of these, theconfigurations and operations of the imaging distance estimating unit110 and the abnormal structure identifying unit 130 are similar to thoseof the first embodiment.

The examination region setting unit 410 includes the levelclassification unit 311, the region dividing unit 312, and a localregion setting unit 411. Of these, the configurations and operations ofthe level classification unit 311 and the region dividing unit 312 aresimilar to those of the third embodiment.

The local region setting unit 411 sets a local region in a region wherea subject at the same level of imaging distance classified by the levelclassification unit 311 is shown. In more detail, the local regionsetting unit 411 includes a distance transform image calculating unit411 a that calculates a distance transform image acquired bytransforming distances from a boundary of the region where the subjectat the imaging distance classified in the same level is shown into pixelvalues, and the local region setting unit 411 sets the local regionbased on the distance transform image.

Next, operations of the image processing apparatus 4 will be described.

FIG. 23 is a flow chart illustrating the operations of the imageprocessing apparatus 4. Steps S11, S12, S14, and S15 illustrated in FIG.23 correspond to those of the first embodiment (see FIG. 2).

At Step S41 subsequent to Step S12, the examination region setting unit410 sets an examination region in an image such that an index indicatinga spread of a distribution of imaging distance of a subject shown in theexamination region is within a given range. FIG. 24 is a flow chartillustrating in detail a process executed by the examination regionsetting unit 410. Further, FIG. 25 is a schematic diagram illustratingthe process executed by the examination region setting unit 410.

At Step S411, the level classification unit 311 classifies values of theimaging distances to the subject shown in the image “M” into a pluralityof given levels.

At subsequent Step S412, the region dividing unit 312 divides the image“M” into each region where the subject at the same level of imagingdistance is shown. Thereby, for example, in FIG. 25, the image “M” isdivided into the four divided regions B1 to B4 correspondingly with thelevel R1 where the imaging distance is equal to or less than r1, thelevel R2 where the imaging distance is in the range of r1 to r2, thelevel R3 where the imaging distance is in the range of r2 to r3, and thelevel R4 where the imaging distance is equal to or greater than r3.

Each of the divided regions B1 to B4 divided like this has an arbitraryshape according to the subject in the image “M”. Therefore, if thesedivided regions B1 to B4 are directly set as examination regions toexecute a process of calculating particular spatial frequency componentsas texture feature data, the necessity of determining whether or not theentire pixels of rectangular regions including the examination regionsare of examination regions for which the particular spatial frequencycomponents are to be calculated is created, and a long period of time isrequired in that process. Therefore, in the fourth embodiment, based onthe divided regions B1 to B4, an examination region is set for a localregion in a region where a target at the same level of imaging distanceis shown.

In detail, at Step S413, the distance transform image calculating unit411 a calculates a distance transform image from a boundary of theregion where the subject at the same level of imaging distance is shownand from a region where an examination region has been already set.

At subsequent Step S414, the local region setting unit 411 sets anexamination region such that a pixel having a maximum value of values inthe distance transform image is a center coordinate of the examinationregion and distances from the center coordinate to an end of theexamination region are less than the maximum value. The pixel having themaximum value of values in the distance transform image is any one ofpixels at midpoints with equal distances from two boundaries, and forexample, in FIG. 25, a pixel P_(i) positioned at a midpoint between aboundary D1 and a boundary D2 of the divided region B2 corresponding tothe level R2 corresponds to that pixel. In that case, the maximum valueof the values of the distance transform image is a distance d_(i) to theboundary D1 or D2 from the pixel P_(i). With the pixel P_(i) at thecenter, an examination region EC_(i) with the maximum value of thediagonal line not exceeding a distance of d_(i)×2 is set. Thereby, in aregion where a subject at the same level of imaging distance is shown, arectangular examination region is able to be set.

At Step S42 subsequent to Step S41, the repeat control unit 220determines whether or not an area of an unexamined region is less thanthe threshold value “A” or whether or not the number of times anexamination region has been set so far is greater than the thresholdvalue “N”. The reason for performing this determination is because ifexamination regions have not been set sufficiently in an image, there isa possibility that accuracy of identification of an abnormality in amicrostructure may be reduced.

If the area of the unexamined region is equal to or greater than thethreshold value “A” and the number of times an examination region hasbeen set is equal to or less than the threshold value “N” (Step S42:No), the repeat control unit 220 determines that further setting of anexamination region is needed, proceeds to Step S41, and causes theexamination region setting unit 410 to execute the setting of anexamination region again. On the contrary, if the area of the unexaminedregion is less than the threshold value “A” or if the number of times anexamination region has been set is greater than the threshold value “N”(Step S42: Yes), the process proceeds to Step S14.

As described above, according to the fourth embodiment, withoutrepeating a process of setting and determining an examination candidateregion or the like, an examination region enabling efficient executionof calculation of a particular spatial frequency component is able to beset. Therefore, the overall process of identifying an abnormality in amicrostructure is able to be speeded up.

Fifth Embodiment

Next, a fifth embodiment of the present invention will be described.

FIG. 26 is a block diagram illustrating a configuration of an imageprocessing apparatus according to the fifth embodiment of the presentinvention. As illustrated in FIG. 26, an image processing apparatus 5according to the fifth embodiment includes a calculating unit 500,instead of the calculating unit 100 illustrated in FIG. 1. Theconfigurations and operations of the respective units of the imageprocessing apparatus 5 other than the calculating unit 500 are similarto those of the first embodiment.

The calculating unit 500 includes the imaging distance estimating unit110, the examination region setting unit 120, an examination regiondeforming unit 510 that deforms an image in an examination region, andthe abnormal structure identifying unit 130. Of these, theconfigurations and operations of the imaging distance estimating unit110, the examination region setting unit 120, and the abnormal structureidentifying unit 130 are similar to those of the first embodiment.

The examination region deforming unit 510 includes a representativeimaging distance acquiring unit 511 that acquires a representativeimaging distance to a subject shown in an examination region and a sizenormalizing unit 512 that normalizes a size of the examination regionaccording to the representative imaging distance.

Next, operations of the image processing apparatus 5 will be described.

FIG. 27 is a flow chart illustrating the operations of the imageprocessing apparatus 5. Steps S11 to S15 illustrated in FIG. 27correspond to those of the first embodiment (see FIG. 2).

At Step S51 subsequent to Step S13, the examination region deformingunit 510 deforms the image in the examination region. As describedabove, in an intraluminal image captured by an endoscope, resolutions ofa microstructure of a mucosal surface differ according to the imagingdistance. Accordingly, in this fifth embodiment, according to theimaging distance to the set examination region, the image in theexamination region is deformed, and a particular spatial frequencycomponent in the deformed image is used as texture feature data toidentify an abnormality in a microstructure, to thereby improve theidentification accuracy.

FIG. 28 is a flow chart illustrating in detail a process (Step S51)executed by the examination region deforming unit 510. Further, FIG. 29is a schematic diagram illustrating a concept of the process executed bythe examination region deforming unit 510.

First, at Step S511, the representative imaging distance acquiring unit511 acquires a representative imaging distance to a subject shown ineach examination region.

At subsequent Step S512, the size normalizing unit 512 normalizes a sizeof the examination region according to the representative imagingdistance. As illustrated in FIG. 29, at a certain imaging distance r₀, acycle of a pixel value change corresponding to particular spatialfrequency enabling identification of an abnormality in a microstructureis assumed to be c₂. As compared to this imaging distance r₀, as theimaging distance becomes shorter, the particular spatial frequencyshifts to a low frequency side and the cycle of the pixel value changebecomes larger (c₁>c₂). On the contrary, as compared to this imagingdistance r₀, as the imaging distance becomes longer, the particularspatial frequency shifts to a high frequency side and the cycle of thepixel value change becomes smaller (c₃<c₂).

Accordingly, as compared to the imaging distance r₀, if the imagingdistance of an examination region EC1 to be processed is shorter (thatis, for a near view portion), by reducing the examination region EC1according to the ratio between the cycles c₁ and c₂, a resolution of thereduced examination region EC1′ is able to be made equal to that for theimaging distance r₀. On the contrary, as compared to the imagingdistance r₀, if the imaging distance of an examination region EC2 to beprocessed is longer (that is, for a distant view portion), by enlargingthe examination region EC2 according to the ratio between the cycles c₂and c₃, a resolution of the enlarged examination region EC2′ is able tobe made equal to that for the imaging distance r₀. That is, byperforming such deforming, at later Step S14, regardless of the imagingdistance of the examination region, the same particular spatialfrequency component is able to be calculated to perform identificationof an abnormality.

As described above, according to the fifth embodiment, by normalizing,based on the representative imaging distance to the subject shown in theexamination region, the size of the examination region, regardless ofimaging distance, by calculating the same particular spatial frequency,an abnormality in a microstructure is able to be identified accurately.

Modified Example 5-1

Next, a modified example 5-1 of the fifth embodiment will be described.

FIG. 30 is a block diagram illustrating a configuration of anexamination region deforming unit according to the modified example 5-1.A calculating unit according to the modified example 5-1 includes anexamination region deforming unit 520 illustrated in FIG. 30, instead ofthe examination region deforming unit 510 illustrated in FIG. 26. Thisexamination region deforming unit 520 includes a three dimensionalcoordinate estimating unit 521 that estimates three dimensionalcoordinates, based on the imaging distance and coordinates in the image,for at least three reference points on a subject shown in an examinationregion, and an image transform unit 522 that performs image transform onthe examination region such that the examination region is transformedinto an image acquired when the examination region in a plane spanned bythe at least three reference points is directly oppositely imaged from agiven distance.

Operations of the calculating unit according to the modified example 5-1as a whole are similar to those illustrated in FIG. 27 and the processat Step S51 is different from that of the fifth embodiment.

FIG. 31 is a flow chart illustrating details of a process (Step S51)executed by the examination region deforming unit 520. Further, FIG. 32Aand FIG. 32B are schematic diagrams illustrating a concept of theprocess executed by the examination region deforming unit 520.

At Step S51 subsequent to Step S13, the examination region deformingunit 520 deforms an image in the examination region. Although in thefifth embodiment, the size of the examination region is normalized, adifference in imaging distance within a given range remaining in theexamination region is not corrected, and influence is generated due to adifference between a resolution of the microstructure of the mucosalsurface with a short imaging distance and a resolution of themicrostructure of the mucosal surface with a long imaging distance inthe same examination region. Therefore, in this modified example 5-1, asillustrated in FIG. 32A, an examination region is deformed as if aregion including three reference points (for example, points P₁, P₂, andP₃) on the mucosal surface 8 in the lumen 7 has been imaged by theendoscope 6 from the front.

In detail, at Step S521, the three dimensional coordinate estimatingunit 521 estimates three dimensional coordinates, based on the imagingdistance and coordinates in the image, for the at least three referencepoints on the subject shown in the examination region. In the actualprocess, arbitrary three pixels away from one another in the examinationregion are selected first. Then, as illustrated in FIG. 32B, forexample, based on imaging distances r₁, r₂, and r₃ to positions(reference points) P₁, P₂, and P₃ on the subject corresponding to thesepixels, a focal distance of the imaging device (for example, theendoscope 6), and coordinates (x₁, y₁), (x₂, y₂), and (x₃, y₃) in theimage corresponding to the positions P₁, P₂, and P₃, three dimensionalcoordinates of the positions P₁, P₂, and P₃ on the subject in acoordinate system with the origin being the imaging device areestimated. The focal distance of the imaging device is determinedbeforehand.

At subsequent Step S522, the image transform unit 522 performs imagetransform on the examination region such that the examination regionbecomes an image acquired when a region on the subject in theexamination region is imaged from the front at a given distance. Thisimage transform may be executed by supposing, for example, thattransform is performed to an image captured at a given focal distancewhile placing a viewpoint at a position away by a given distance towardsa normal direction of a plane PL passing through the positions P₁, P₂,and P₃ on the subject from gravity centers of the positions P₁, P₂, andP₃ on the plane PL (or gravity centers of regions on the subjectcorresponding to the examination regions).

As described above, according to the modified example 5-1, a differencein imaging distance in an examination region is able to be corrected.That is, in a transformed image, a variation in distance between eachposition in a region on a subject corresponding to each pixel positionin the same examination region and a position of an imaging device isable to be decreased from that in the image that has not beentransformed. Thereby, a difference between a resolution of amicrostructure of a mucosal surface with a near imaging distance and aresolution of the microstructure of the mucosal surface with a distantimaging distance in the same examination region is able to be decreased.Therefore, by using, as texture feature data, a particular spatialfrequency component in an image transformed as above, accuracy ofidentification of an abnormality in a microstructure is able to beimproved further.

As described above, according to the first to fifth embodiments andtheir modified examples, since the examination region is set in theimage such that the index indicating the spread of the distribution ofthe imaging distance of the subject shown in the examination region iswithin the given range, and the texture feature data allowingidentification of an abnormality in the microstructure of the subjectshown in the examination region are used for each examination region, tothereby identify whether or not the microstructure of the subject shownin the examination region is abnormal, even if a difference is caused inresolution of a microstructure of a mucosal surface shown in an imagedue to a difference in imaging distance, an abnormality in themicrostructure of the mucosal surface is able to be identifiedaccurately.

In the above described first to fifth embodiments and their modifiedexamples, as an example of the texture feature data, the spatialfrequency component numerically expressing the frequency characteristicsof the texture is used, but statistical feature data of the texture maybe used instead. The statistical feature data of the texture may befound by using a co-occurrence matrix of pixel values. Specifically, bya co-occurrence matrix, from values of a pixel pair at two positionsaway from each other in an image, statistics (feature data) representingcharacteristics, such as uniformity, directionality, contrast, and thelike of the pixel values are able to be found (reference: “Digital ImageProcessing” by CG-ARTS Society, pages 194 to 195 (“Texture of Region”)).

The image processing apparatuses according to the above described firstto fifth embodiments and their modified examples may be realized byexecuting an image processing program recorded in a recording device bya computer system, such as a personal computer or a work station.Further, such a computer system may be used by being connected toanother computer system or a device, such as a server, via a localregion network/wide area network (LAN/WAN), or a public network, such asthe Internet. In this case, the image processing apparatuses accordingto the first to third embodiments may acquire image data of intraluminalimages via these networks, output image processing results to variousoutput devices (such as viewers and printers) connected via thesenetworks, or store the image processing results in storage devices(recording devices and reading devices thereof, or the like) connectedvia these networks.

The present invention is not limited to the first to fifth embodimentsand the modified examples thereof, and various inventions may be formedby combining as appropriate a plurality of structural elements disclosedin the respective embodiments and modified examples. For example,formation by excluding some of the structural elements from the wholestructural elements illustrated in the respective embodiments andmodified examples may be made, or formation by combining as appropriatethe structural elements illustrated in the different embodiments andmodified examples may be made.

Additional advantages and modifications will readily occur to thoseskilled in the art. Therefore, the invention in its broader aspects isnot limited to the specific details and representative embodiments shownand described herein. Accordingly, various modifications may be madewithout departing from the spirit or scope of the general inventiveconcept as defined by the appended claims and their equivalents.

What is claimed is:
 1. An image processing apparatus, comprising: animaging distance estimating unit configured to estimate an imagingdistance to a subject shown in an image; an examination region settingunit configured to set an examination region in the image such that anindex indicating a spread of a distribution of imaging distances to thesubject shown in the examination region is within a given range; and anabnormal structure identifying unit configured to identify whether ornot a microstructure of the subject shown in the examination region isabnormal, by using texture feature data that enables identification ofan abnormality in the microstructure of the subject shown in theexamination region, the texture feature data being specified accordingthe examination region.
 2. The image processing apparatus according toclaim 1, wherein the examination region setting unit includes: acandidate region setting unit configured to set an examination candidateregion in the image; and a region determining unit configured todetermine an examination region based on imaging distance information ofthe subject shown in the examination candidate region.
 3. The imageprocessing apparatus according to claim 2, wherein the candidate regionsetting unit includes a representative imaging distance acquiring unitconfigured to acquire a representative imaging distance to the subjectshown at a position where the examination candidate region is to be set,and the candidate region setting unit is configured to set theexamination candidate region of a size according to the representativeimaging distance.
 4. The image processing apparatus according to claim2, wherein the candidate region setting unit includes a representativeimaging distance gradient calculating unit configured to calculate arepresentative imaging distance gradient of the subject shown at aposition where the examination candidate region is to be set, and thecandidate region setting unit is configured to set the examinationcandidate region of a size according to the representative imagingdistance gradient.
 5. The image processing apparatus according to claim2, wherein the region determining unit includes an imaging distancerange calculating unit configured to calculate a distribution range ofthe imaging distances to the subject shown in the examination candidateregion, and the region determining unit is configured to determine theexamination candidate region with the distribution range equal to orless than a given threshold value, to be the examination region.
 6. Theimage processing apparatus according to claim 2, wherein the regiondetermining unit includes an imaging distance variance calculating unitconfigured to calculate a variance of the imaging distances to thesubject shown in the examination candidate region, and the regiondetermining unit is configured to determine the examination candidateregion with the variance equal to or less than a given threshold value,to be the examination region.
 7. The image processing apparatusaccording to claim 1, wherein the examination region setting unitincludes: a level classification unit configured to classify values ofthe imaging distances into one level or a plurality of levels; and aregion dividing unit configured to divide the image into one region or aplurality of regions, for each region where the subject at a same levelof imaging distance is shown, and the examination region setting unit isconfigured to set each of the one region or the plurality of regionsacquired by the region dividing unit, as an individual examinationregion.
 8. The image processing apparatus according to claim 7, whereinthe examination region setting unit further includes a local regionsetting unit configured to set a local region in the region where thesubject at the same level of imaging distance is shown, and theexamination region setting unit is configured to set the local region asthe examination region.
 9. The image processing apparatus according toclaim 8, wherein the local region setting unit includes a distancetransform image calculating unit configured to calculate a distancetransform image acquired by transforming a distance from a boundary ofthe region where the subject at the same level of imaging distance isshown, into a pixel value, and the local region setting unit isconfigured to set the local region based on the distance transformimage.
 10. The image processing apparatus according to claim 1, furthercomprising a repeat control unit configured to perform control ofrepeating a process by the examination region setting unit with respectto a region where the examination region has not been set.
 11. The imageprocessing apparatus according to claim 10, wherein the repeat controlunit is configured to change the process executed by the examinationregion setting unit according to the repeating of the process.
 12. Theimage processing apparatus according to claim 1, wherein the abnormalstructure identifying unit includes: a particular frequency componentcalculating unit configured to calculate, as the texture feature data, aparticular spatial frequency component that enables identification of anabnormality in the microstructure of the subject shown in theexamination region; and a statistical classification unit configured toperform statistical classification based on the particular spatialfrequency component.
 13. The image processing apparatus according toclaim 12, wherein the abnormal structure identifying unit furtherincludes a particular wavelength component selecting unit configured toselect a particular wavelength component specified according to a degreeof absorption or scattering in a living body, and the particularfrequency component calculating unit is configured to calculate theparticular spatial frequency component with respect to the particularwavelength component.
 14. The image processing apparatus according toclaim 12, wherein the abnormal structure identifying unit furtherincludes an inter-particular-wavelength ratio calculating unitconfigured to calculate a ratio between particular wavelength componentshaving different degrees of absorption or scattering in a living body,and the particular frequency component calculating unit is configured tocalculate the particular spatial frequency component with respect to theratio between the particular wavelength components.
 15. The imageprocessing apparatus according to claim 12, wherein the abnormalstructure identifying unit further includes a representative imagingdistance acquiring unit configured to acquire a representative imagingdistance to the subject shown in the examination region, and theparticular frequency component calculating unit is configured to specifya frequency of the particular spatial frequency component according tothe representative imaging distance and to calculate the particularspatial frequency component.
 16. The image processing apparatusaccording to claim 1, further comprising an examination region deformingunit configured to deform an image in the examination region.
 17. Theimage processing apparatus according to claim 16, wherein theexamination region deforming unit includes: a representative imagingdistance acquiring unit configured to acquire a representative imagingdistance to the subject shown in the examination region; and a sizenormalizing unit configured to normalize a size of the examinationregion according to the representative imaging distance, and theabnormal structure identifying unit is configured to identify whether ornot the microstructure of the subject shown in the examination region isabnormal, by using the texture feature data that enables identificationof an abnormality in the microstructure of the subject shown in theexamination region, the texture feature data being specified accordingthe examination region having the size normalized by the sizenormalizing unit.
 18. The image processing apparatus according to claim16, wherein the examination region deforming unit includes: a threedimensional coordinate estimating unit configured to estimate threedimensional coordinates for at least three reference points on thesubject shown in the examination region, based on the imaging distanceand coordinates of corresponding pixels in the image; and an imagetransform unit configured to perform image transform on the examinationregion such that an image of the examination region is acquired byimaging the examination region in a plane passing through the at leastthree reference points from a front at a given distance.
 19. The imageprocessing apparatus according to claim 1, wherein the image is formedof a plurality of wavelength components, and the imaging distanceestimating unit includes a low absorbance wavelength selecting unitconfigured to select, from the plurality of wavelength components, a lowabsorbance wavelength component that is a wavelength component with alowest degree of absorption or scattering in a living body, and theimaging distance estimating unit is configured to estimate the imagingdistance to the subject shown in the image, based on the low absorbancewavelength component.
 20. The image processing apparatus according toclaim 1, further comprising a non-examination target region excludingunit configured to exclude a non-examination target region included inthe image.
 21. The image processing apparatus according to claim 20,wherein the non-examination target region is any one of a dark region, abright region, and a region where a residue or bubble is shown, in theimage.
 22. An image processing method comprising: an imaging distanceestimating step of estimating an imaging distance to a subject shown inan image; an examination region setting step of setting an examinationregion in the image such that an index indicating a spread of adistribution of imaging distances to the subject shown in theexamination region is within a given range; and an abnormal structureidentifying step of identifying whether or not a microstructure of thesubject shown in the examination region is abnormal, by using texturefeature data that enables identification of an abnormality in themicrostructure of the subject shown in the examination region, thetexture feature data being specified according the examination region.23. A computer-readable recording device with an executable programstored thereon, the program instructing a processor to perform: animaging distance estimating step of estimating an imaging distance to asubject shown in an image; an examination region setting step of settingan examination region in the image such that an index indicating aspread of a distribution of imaging distances to the subject shown inthe examination region is within a given range; and an abnormalstructure identifying step of identifying whether or not amicrostructure of the subject shown in the examination region isabnormal, by using texture feature data that enables identification ofan abnormality in the microstructure of the subject shown in theexamination region, the texture feature data being specified accordingthe examination region.